1. Field of the Invention
The present invention relates to a synchronization system for achieving real-time encoding, transmission, and decoding of data such as video data and audio data, and a method for synchronizing an encoder and a decoder in the synchronization system.
2. Description of the Related Art
In a system including an encoder for compressing data such as video data and audio data in real time, and a decoder for decompressing the data in real time, the encoded data is temporarily stored in a buffer before transmission to prevent data excess or shortage at the decoder end. In the system, the encoder and the decoder are located some distance away from each other, and the data is transmitted in real time from the encoder to the decoder.
In such a system, however, if the encoder at the transmission end and the decoder at the reception end are controlled totally independently, it may cause problems as below sooner or later.
For example, in a case where the decoder is faster in data processing speed than the encoder, the data stored in the buffer may end up being totally decoded sometime soon, causing buffer underflow resulting from no data to be decoded.
On the other hand, if the encoder is faster in data processing speed than the decoder, the buffer may end up overflowing with the data, causing buffer overflow and resulting in data loss.
In view thereof, for correct data decoding, the encoder and decoder have to be synchronized with each other.
To establish synchronization between the encoder and decoder, in an MPEG-2 (Moving Picture Experts Group Phase 2) system, MPEG2-PS (Program Stream) uses SCR (System Clock Reference) and MPEG2-TS (Transport Stream) uses PCR (Program Clock Reference) as reference time information. Here, the reference time information is an index value for reference time. This favorably leads to correct data decoding without causing overflow and underflow of a buffer used for temporary storage.
To be specific, at the time of encoding in the MPEG-2 system, SCR or PCR is generated based on STC (System Time Clock) being an output value of an STC counter for counting encoder clocks. Then, as shown in the upper part of FIG. 11, the resulting SCR or PCR is inserted into an encoding stream at regular time intervals.
At the decoder end, thus inserted SCR or PCR is detected for comparison with STC (output value of the STC counter) at the decoder end as shown in the lower part of FIG. 11. If the comparison result tells that an error therebetween is equal to a certain value or larger, the value of SCR or PCR is loaded to the STC counter so as to correct the STC in value. Thus, the time interval between STCs at the decoder end remains constant except when STC is corrected in value by SCR or PCR.
Further, at the time of encoding in the MPEG-2 system, also inserted into the encoding stream are DTS (Decoding Time Stamp) and PTS (Presentation Time Stamp). Here, DTS is time management information about decoding, and PTS (Presentation Time Stamp) is time management information about reproduction output. At the decoder end, data decoding and output are managed through comparison among DTS, PTS, and STC.
The problem here is that, if the transmission path of the encoding stream jitters, this causes jittering also to SCR or PCR itself being the reference time information. As a result, synchronization control as above cannot be correctly carried out using SCR or PCR.
Considered now is a case exemplified in FIGS. 12A to 12D. That is, an encoding stream including SCRs or PCRs inserted thereto at regular time intervals as shown in FIG. 12A is packetized for transmission as shown in FIG. 12C. For transmission, used are a pack size and a transmission timing as shown in FIG. 12B. Here, the communication protocol to be used is TCP/IP (Transmission Control Protocol/Internet Protocol), for example. In such a case, if packet loss occurs in the transmission path, SCRs or PCRs in the encoding stream may not be received at the same time intervals at the receiver end as shown by D1, D2, and D3 in FIG. 12D. As a result, time correction cannot be done with accuracy.
As a solution of such a problem, JP-A-11-215494 and JP-A-2000-92130 disclose a method for time correction through addition of a timestamp (time information) to packets in a transmission path. Here, used for the transmission path is IEEE1394 serial interface, for example.
With this method, an encoding stream including SCRs or PCRs inserted threto at regular time intervals as shown in FIG. 13A is packetized for transmission as shown in FIG. 13C. For transmission, used are a pack size and a transmission timing as shown in FIG. 13B. At the time of transmission, the timestamp for the transmission path is added to the corresponding packet.
Thus, even if the encoding stream just received at the receiver end becomes the one as shown in FIG. 13D due to packet loss, for example, the timestamp is used for time control of the encoding stream. In this manner, as denoted by D0 in FIG. 13E, SCRs or PCRs in the encoding stream can be located at the same time intervals, leading to correct data decoding.
With such a method of adding a timestamp to packets in the transmission path exemplarily using IEEE1394 serial interface, however, there arises also a problem of considerable increase of the entire system cost.
As a solution of such a problem, JP-A-2002-165148 notes the fact that the data storage amount in a buffer changes depending on a difference between the data processing speed at the encoder end and that at the decoder end. That is, if the data processing speed at the encoder end is faster than that at the decoder end, the data storage amount in a buffer increases. Conversely, if the data processing speed at the decoder end is faster than that at the encoder end, the data storage amount in a buffer decreases. Accordingly, disclosed in the publication is monitoring the data storage amount in a buffer, and when the amount exceeds an upper threshold value, increasing the clock frequency for decoding and the data processing speed at the decoder end. When the amount falls short of a lower threshold value, decreasing the clock frequency for decoding and the data processing speed at the decoder end.
[Patent Document 1]
JP-A-11-215494
[Patent Document 2]
JP-A-2000-92130
[Patent Document 3]
JP-A-2002-165148
The issue here is that, for encoding in the MPEG system, CBR (Constant Bit Rate) and VBR (Variable Bit Rate) are both applicable, and VBR is popular therefor.
When VBR is used, the data amount shows a change per a second. As a result, the data storage amount in a buffer does not respond to a difference between the data processing speed at the encoder end and that of the decoder end.
Thus, with the method of JP-A-2002-165148 for detecting a difference of the data processing speed between the encoder and decoder ends using the data storage amount of a buffer, the difference detection cannot be performed with accuracy, resulting in failure to synchronize the decoder end to the encoder end with reliability.
Even if CBR is used, with the method, the difference of the data processing speed between the encoder and decoder ends cannot still be accurately detected from the data storage amount of a buffer. This is because, exemplarily in a case of wirelessly transmitting an encoding stream, if packet loss or data garbling occurs during transmission, the corresponding data is not stored in the buffer.